Improve testing against PHPUnit version 10, 11, and 12 #116
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
delatrie
force-pushed
the
phpunit-deps-range-fix
branch
from
488a5f3 to
ee13265
Compare
delatrie
force-pushed
the
phpunit-deps-range-fix
branch
from
ee13265 to
f235501
Compare
delatrie
force-pushed
the
phpunit-deps-range-fix
branch
from
fe5e99c to
6139eb3
Compare
baev
approved these changes
Apr 17, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
#115 has added support for PHP 8.4 and PHPUnit 11 and 12.
This PR continues the efforts to
allure-phpunitrequires.Supported PHP versions
We support all versions currently supported by the PHP devs, see https://www.php.net/supported-versions. Currently, those are
^8.1 || ^8.2 || ^8.3 || ^8.4.Supported PHPUnit versions
Currently, we aim to support PHPUnit 10, 11, and 12. Due to the requirements of
brianium/paratest(our dev dependency), we can't install the following PHPUnit versions:Since we use the same PHPUnit installation for running our tests and the Allure adapter for PHPUnit, we can't test against the versions above either in CI or on personal machines. If a bug is reported with one of those versions, we cannot write a test that reproduces it without tweaking the dependency spec.
With that in mind, this PR sets the declared supported versions of PHPUnit to
^10.0.5 || ^11 || ^12.0.1.PHPUnit versions in the CI matrix
Previously, we only tested against PHPUnit
^10with and without--prefer-lowest.After versions
^11and^12were added torequire, that testing scheme became incomplete: it skipped PHPUnit 11.This PR adds a PHPUnit version dimension to the testing CI job, which enables testing against the whole range of the supported PHPUnit versions.
There are some incompatibilities between PHP, PHPUnit, and ParaTest (see
includeandexcludeinbuild.yml).Additionally, we have some tests that mock
PHPUnit\Event\Code\Test, which isreadonlystarting from PHPUnit 11.0.0. Those tests fail when run with PHPUnit 11.0 and 11.1. One solution would be skipping the tests if mocking fails, but for now, the PR further constrains PHPUnit 11 to^11.2in the CI workflow.The XML schema for the legacy (v10) PHPUnit config files was reverted to
https://schema.phpunit.de/10.0/phpunit.xsd. We are now using a PHP script to get the PHPUnit version at runtime and choose the correct config file based on it.GarbageCollectorStatuscompatibilityPHPUnit\Event\Telemetry\GarbageCollectorStatuswas introduced in PHPUnit 10.1.0. Its constructor requires four extra parameters starting from 10.3.0. The PR fixesEventTestTraitto account for all three cases by checking the number of constructor parameters withReflectionClass.#[CoversClass]and#[CoversTrait]We use
#[CoversClass]to indicate to PHPUnit that onlyExceptionDetailsTrait's methods should be marked as covered by the tests inExceptionDetailsTraitTest. Using traits with#[CoversClass]was an undocumented (and untested) feature of PHPUnit, which became broken at some point. Then,#[CoversTrait]was introduced in 11.2.0, and PHPUnit's devs advised to use it with traits instead of#[CoversClass].The PR refactors
ExceptionDetailsTraitTestto be conditionally defined with the correct attribute, which supports both old and new mechanisms depending on whetherPHPUnit\Framework\Attributes\CoversTraitexists or not.In 11.4.0,
#[CoversTrait]was declared as deprecated: the devs had discovered that#[CoversClass]also covers the target's traits and decided the new attribute wasn't needed in the first place. They reinstated it in 11.5.4 and kept it in version 12 to target traits without targeting the surrounding class. See the discussion here.Note
We are ignoring
PSR1.Classes.ClassDeclaration.MultipleClassesinExceptionDetailsTraitTest.phpbecause it essentially declares a single class used outside.Psalm fixes
The
0.19.4release ofpsalm/plugin-phpunithas introduced new checks on data providers for PHPUnit tests. The ones we have inDataProviderTest.phpfailed them (see here).The PR applies the correct type annotations to the data provider functions, which fixes the issues.
Dependency updates
phpunit/phpunitfromrequire-dev.brianium/paratestfrom^7.3to^7.vimeo/psalmto^6.10.psalm/plugin-phpunitto^0.19.5.PHPUnit configuration update
The PR bumps the XML schema of the default PHPUnit config files we use to
https://schema.phpunit.de/12.1/phpunit.xsd.Ideally, we should use a separate config file for each major PHPUnit version we test against. However, in our case, the config file is compatible with versions 11 and 12.